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1 

INTRODUCTION 


This  Paper  contains  excerpts  from  "Interceptor  Development  Engineering  Cost 
Research",  CR-0616,  Tecolote  Research,  Inc.,  February  1993.  Proprietary  Data  has  been 
removed  so  that  it  could  be  presented  at  the  DoD  Cost  Analysis  Symposium.  This  paper 
presents  the  techniques  utilized  in  this  cost  research  effort  and  its  results. 
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OBJECTIVES 


The  objective  of  this  effort  was  to  examine  Development  Engineering  of  Missile  Air 
Vehicles  through  review  of  existing  estimating  methodologies  and  data  research,  collection, 
normalization,  and  development  of  recommended  methods/CERs  for  use  in  USASSDC 
estimates.  Development  Engineering  is  typically  a  significant  cost  driver  of  the  R&D  phase. 
In  fact,  the  Development  Engineering  cell  alone  may  constitute  as  much  as  one-third  of  the 
R&D  phase  costs.  Development  Engineering  may  sometimes  be  the  largest  cost  element  in 
R&D.  Given  the  magnitude  of  Development  Engineering,  the  development  of  appropriate 
estimating  techniques  for  DE  is  of  proper  concern. 


The  definitions  of  Development  Engineering  is  consistent  in  cost  estimating  guidance. 
The  DA  PAM  11-2  Research  and  Development  Cost  Guide  for  Material  Systems,  the  DCA- 
P-92(R)  Instructions  for  Reformatting  the  BCE/ICE,  and  the  new  Cost  Analysis  Manual 
define  Development  Engineering  essentially  the  same.  The  latest  definition  is: 

"This  element  includes  the  costs  of  study,  analysis,  design  development, 
evaluation,  testing,  and  redesign  for  the  system  component(s)  during  the 
system  development  efforts.  It  includes  the  design  efforts  of  preparing 
specifications,  engineering  drawings,  parts  lists,  wiring  diagrams,  test 
planning  and  scheduling,  analysis  of  test  results,  data  reduction,  report 
preparations  and  establishment  of  reliability,  maintainability,  and  quality 
assurance  control  requirements,  It  also  includes  the  costs  of  raw  and  semi- 
fabricated  material  plus  purchased  parts  consumed  in  the  performance  of 
component  engineering  efforts.  Also  included  is  engineering  test  equipment 
such  as  oscilloscopes,  transducers,  recorders,  radio  transmitters,  converters, 
discriminators,  receivers,  and  other  equipment  required  to  accomplish  the 
engineering  function  for  the  specified  system  components.  This  element  also 
includes  the  engineering  efforts  in  support  of  pre-planned  product 
improvements.  Excluded  from  this  element  are  the  engineering  efforts 
(producibility  engineering  and  planning)  to  ensure  producibility  of  the  item  or 
system  prior  to  quantity  procurement." 
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RESEARCH 


After  reviewing  existing  Development  Engineering  CERs  and  studying  the  estimating 
challenges  of  this  element,  it  is  apparent  that  schedule  must  play  an  important  role  in  the  cost 
of  air  vehicle  development 

Development  Engineering  cost  is  hypothesized  to  be  a  function  of  the  complexity  of 
the  item  being  developed  and  the  time  it  takes  to  develop  it.  The  complexity  of  the  item  may 
be  a  function  of  technical  parameters  which  may  be  represented  by  unit  cost,  which  in  turn, 
may  be  used  as  a  surrogate  of  complexity.  Many  existing  Development  Engineering  CERs 
used  some  variation  of  prototype  cost  and  that  is  a  value  that  is  readily  available  to  the 
estimator.  Other  complexity  variables  may  be  the  leverage  a  program  receives  from  previous 
or  parallel  programs  or  programmatic  complexities  such  as  the  number  of  major 
subcontractors,  etc. 

Development  Engineering  cost  is  also  hypothesized  to  be  a  function  of  schedule.  The 
total  duration  of  the  development  program  is  a  compelling  cost  driver:  some  minimum  cadre 
of  designers  should  be  expected  to  be  applied  for  the  duration  of  the  development  contract. 
However  the  size  of  this  staff  may  in  fact  vary  over  time  especially  as  issues  are  resolved 
during  the  test  program  and  as  the  design  becomes  more  stable. 

3.1  HYPOTHESIS 

The  hypothesis  developed  for  this  study  is  tha*  Development  Engineering  cost  may  be 
represented  in  two  parts.  The  first  being  a  technical  component.  The  technical  component  is 
hypothesized  as  an  equation  that  will  compute  the  average  size  of  the  staff  (or  monthly 
expenditure  rate)  that  it  takes  to  accomplish  the  basic  design  of  the  air  vehicle.  This  equation 
will  have  independent  variables  such  as  hardware  unit  cost  or  physical  parameters.  It  may  also 
have  variables  that  allow  the  addressing  of  program  leverage  or  the  programmatic  complexity. 
The  second  part  is  the  schedule  component.  This  is  hypothesized  as  being  a  series  of  terms 
where  a  portion  of  the  technical  component  is  applied  to  periods  of  time  between  milestones. 
The  product  of  these  components  results  in  total  Development  Engineering  costs. 


The  hypothesis  is  further  refined  by  stating  that  the  cost  of  development  is  related  to 
the  monthly  expenditure  rate  (which  is,  in  turn  related  to  the  size  of  the  design  staff)  and 
duration.  Further,  the  monthly  expenditure  rate  is  different  at  different  phases  of  the 
development  program  and  peaks  in  the  period  prior  to  Critical  Design  Review  (CDR).  It  is 
further  hypothesized  that  the  expenditure  rate  in  other  periods  is  related  to  the  peak  in  a 
predictable  manner  and  that  the  peak  expenditure  rate  is  related  to  technical  and  programmatic 
complexity. 

Summarizing  the  hypothesis  algebraically,  Development  Engineering  is  related  to 
expenditure  rate  and  duration, 

DE$  =  f {rate,  duration), 

and  that  the  peak  monthly  expenditure  rate,  Rmax  is  related  to  air  vehicle  complexity, 

/?  max  =  f  ( complexity ) , 

and  that  the  expenditure  rate  during  each  period  i  may  be  expressed  as 


Ri  —  (h  *  R  mix  . 


Now  if  observed  data  can  be  utilized  to  determine  a  set  of  coefficient  and  an  equation  to 
predict  Rmax  can  be  determined,  then  the  cost  of  Development  Engineering  may  be  expressed 

as 


DE%  —  a<*R  n»x  *  Di 

or 

£>£$  = 


where  D/  is  the  duration  of  each  period.  This  is  illustrated  graphically  in  Figure  3-1. 


Figure  3-1 
Hypothesis 


The  bulk  of  the  task  centered  around  two  efforts.  First  to  determine  the  set  of 
coefficients  0j  and  second  to  determine  the  equation  for  Rmax. 

3.2  DATA  COLLECTION 

Data  was  sought  for  missile  system  air  vehicle  development  contracts  for  all  phases: 
Engineering  and  Manufacturing  Development  (EMD),  Demonstration  and  Validation  (D&V), 
Product  Improvement  Programs  (PIP),  and  experiments.  In  particular,  recent  USASSDC 
experiments  and  USASSDC  Demonstration  and  Validation  (D&V)  programs  cost  data  was 
sought.  The  most  desirable  data  was  monthly  cost  reports.  Contractor  Cost  Data  Reports 
(CCDRs)  were  preferred  over  Cost  Performance  Reports  (CPRs)  because  the  non-recurring 
engineering  was  visible.  Monthly  data  for  fifteen  (15)  systems  were  successfully  obtained. 
For  twenty-two  (22)  systems,  the  final  (or  latest)  available  cost  report  was  used.  An 
additional  nineteen  (19)  data  points  were  used  from  prior  studies.  Source  cost  reports  were 
not  available  for  this  last  group  though  through  conversation  with  those  investigators  the 
author  has  confidence  that  those  data  points  weie  normalized  correctly.  The  sources  are 
summarized  as: 


Monthly  CPRs  9 

Monthly  CCDRs  6 

Single  CPRs  10 

Single  CCDR  9 

CEAC  Infoarch  6 

Prior  Study  13 

Other  2 


3.3  NORMALIZATION 

Normalization  of  the  data  was  done  in  two  steps.  First,  the  effects  of  inflation  were 
removed  using  the  DoD  indices  contained  in  the  19  May  92  inflation  letter  to  express  the 
values  in  FY88  dollars.  Second,  Development  Engineering  cost  was  isolated.  As  stated 
above,  the  CCDRs  did  not  require  mapping.  In  using  CCDR  data,  the  non-recurring 
engineering  was  equated  with  Development  Engineering.  In  the  CPRs,  many  systems  had 
separate  line  items  for  prototype  fabrication  and  for  design.  Design  was  equated  with 
Development  Engineering.  However  some  system's  CPRs  contained  accounts  for  Prime 
Mission  Equipment  (PME)  that  was  comprised  of  Development  Engineering  and  Prototype 


Manufacturing  together.  For  these  systems,  Development  Engineering  was  isolated  by  taking 
manufacturing  cost  found  on  the  functional  page  and  subtracting  that  value  from  the  PME 
cost  and  equating  the  balance  with  Development  Engineering.  For  obtaining  normalized 
hardware  unit  costs,  learning  curves  for  production  units  are  assumed  at  90%,  EMD  units  at 
95%,  and  D&V  units  at  100%.  EMD  to  production  step  function  is  assumed  at  1.66.  All 
costs  contained  in  this  paper  (unless  otherwise  noted)  are  in  millions  of  FY88  dollars  and 
include  G&A  and  fee.  Each  r^  that  is  presented  in  this  paper  is  adjusted  for  degrees  of 
freedom  and  each  Standard  Error  (SE)  is  in  unit  space. 

3.4  DATABASE 

The  database  for  evaluating  the  hypothesis  is  exemplified  in  the  sample  data  point 
shown  in  Figure  3-2.  Each  parameter  in  Figure  3-2  is  defined  as  follows. 


Development  Engineering  cost  is  as  defined  in  DA  PAM  11-2,  P-92,  and  the  Cost 
Analysis  Guide.  This  is  contract  cost  for  subcontractors  as  well  as  prime;  Government  cost  is 
excluded.  Where  possible,  software  costs  are  excluded.  Prototype  Manufacturing  is  also  as 
defined  by  DA  PAM  1 1-2,  P-92,  and  the  Cost  Analysis  Guide.  The  cost  of  Fust  prototype  is 
the  theoretical  cost  of  the  First  prototype  assuming  slopes  as  defined  above  by  program  phase. 

Design  Basis  is  a  new  term  introduced  for  this  study.  It  is  the  Development 
Engineering  cost  incurred  in  the  period  from  Preliminary  Design  Review  (PDR)  to  CDR. 
That  is,  the  amount  of  effort  to  actually  accomplish  the  air  vehicle  design.  Peak  design  effort 
is  the  average  monthly  expenditure  rate  observed  during  this  period.  Prototype  Quantity  us 
the  number  of  Air  Vehicles  assembled  during  the  contract.  Test  quantity  is  the  number  of  Air 
Vehicles  flown.  Sled  tests  and  chamber  tests  are  not  included.  The  R&D  Slope  Basis  is  the 
cumulative  average  learning  curve  slope  percentage  used  for  each  data  point  which  varies  by 
program  phase.  The  Duration  is  the  number  of  months  from  contract  award  to  contract  end. 
Missile  Weight  is  expressed  in  pounds  and  is  the  launch  weight  for  the  entire  Air  Vehicle.  The 
Software  Cost  shown  in  the  final  column  are  those  costs  that  were  specifically  available  on  the 
cost  report  and  could  be  confidently  associated  with  the  Air  Vehicle. 


Figure  3-3  presents  an  example  regarding  the  expenditures  during  specified  periods  of 


time.  For  each  system,  three  rows  of  va! 


nor  n  ra  rUnum 

uvu  Miv  Jiiv/rvii. 


The  first  row  is  die  iiumuci  of  inumns 


per  period,  the  second  is  the  total  cost  reported  for  that  period,  and  the  third  is  the  observed 
average  cost  per  month  during  that  period. 


PARAMETER 

VALUE  I 

r 

SOURCE 

CPR 

DEVELOPMENT  ENGINEERING  (FY88M$) 

169.841 

PROTOTYPE  MANUFACTURING  (FY88M$) 

146.484 

PEAK  (FY88M$) 

4.257 

DESIGN  BASIS  (FY88MS) 

63.857 

PROTOTYPE  QUANTITY 

34 

TEST  QUANTITY 

28 

PROTOTYPE  CFU  (FY88M$) 

5.593 

R&D  SLOPE  BASIS 

0.95 

PHASE 

EMD 

TOTAL  DURATION  (MO) 

67 

MISSILE  WEIGHT  (LBS.) 

16400 

CONTRACT  AWARD 

Feb-79 

PDR 

Jan-80 

CDR 

Apr-81 

FIRST  PROTOTYPE 

Dec-81 

FIRST  TEST 

Apr- 8  2 

LAST  TEST 

Aug-83 

FOLLOW-ON 

Jun-82 

CONTRACT  END 

Sep-84 

Figure  3-2 
Sample  Data  Point 


3.5  SCHEDULE  COMPONENT  DEVELOPMENT 

Actual  milestone  dates  were  collected  for  the  systems  under  study  in  order  to 
determine  the  expenditure  rate  by  period  from  the  monthly  data.  These  dates  were  obtained 
from  project  offices,  Selected  Acquisition  Reports  (SARs),  fact  books,  prior  studies,  CPR 
variance  notes,  and  public  literature.  Some  actual  milestone  dates  were  missing.  While  the 
focus  of  this  task  is  not  to  develop  schedule  estimating  relationships,  it  was  necessary  to 
estimate  missing  milestones.  This  was  accomplished  by  computing  each  milestone's  median 


Award 

to 

PDR 

PDR 

to 

CDR 

— 

m 

Last 
Flight  to 
End 

Sample  System 

Months 

12 

15 

12 

16 

13 

Period  Cost 

26747 

63857 

34805 

37020 

7965 

1  Cost  Per  Month 

2229 

4257 

2900 

2314 

613 

Figure  3-3 

Sample  Expenditures  by  Period 

percentage  of  the  total  duration.  These  median  percentages  were  applied  to  each  system,  and 
from  the  total  duration  of  each,  the  missing  dates  were  computed.  The  observed  median 
percentages  are: 


Contract  award  to  PDR  0.14 

PDR  to  CDR  0.26 

CDR  to  1st  Flight  0.16 

1  st  Flight  to  Last  Flight  0.24 

Last  Flight  to  Contract  End  0.20 


The  dates  are  the  milestones  at  which  certain  events  took  place.  The  dates  that  were 
of  interest  to  this  study  were  those  that  should  signal  a  change  in  the  scope  of  the 
Development  Engineering  effort.  Contract  award  is  simply  the  beginning  of  the  design  effort. 
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Some  staff  is  brought  together  to  begin  design  of  the  system  and  mature  it  sufficiently  for  a 
PDR  to  take  place.  PDRs  are  conducted  after  top  level  design  efforts  are  completed,  but  prior 
to  the  start  of  detailed  design.  A  completed  PDR  represents  approval  to  begin  detailed  design 
and  the  Development  Engineering  effort  begins  in  earnest 

The  heaviest  design  phase  culminates  in  the  CDR.  CDR  is  conducted  before  release  of 
design  for  fabrication.  This  signals  completion  of  the  bulk  of  the  new  design  effort  and  is 
sometimes  defined  as  drawings  being  90%  complete. 

After  CDR  it  would  be  expected  that  the  size  of  the  design  staff  would  begin  to 
decline.  The  next  period  would  end  with  the  delivery  of  the  first  air  vehicle  to  the  launch  site 
for  government  acceptance.  During  this  period  the  Development  Engineering  effort  continues 
in  support  of  the  fabrication  of  the  prototype.  Upon  completion  of  one  prototype  it  could  be 
expected  that  many  design  issues  have  been  resolved  and  that  a  further  decrease  in  the  design 
staff  would  be  forthcoming.  Note  that  only  Development  Engineering  is  being  addressed;  the 
same  engineers  may  take  on  more  systems  engineering  or  test  engineering  functions  such  that 
the  total  engineering  staff  assigned  to  a  given  contract  may  perform  different  tasks.  The  next 
milestone  is  the  date  of  the  first  flight  test.  Upon  success  of  the  first  flight  it  could  be 
expected  that  more  design  issues  have  been  addressed  and  that  the  design  effoit  should 
decrease.  From  the  data  set  it  was  observed  that  many  systems  held  CDR  after  the  test 
program  began.  The  last  test  is  the  last  flight  of  prototype  hardware  accomplished  during  the 
contract.  At  this  milestone  the  majority  of  the  design  issues  have  been  resolved  and  the 
Development  Engineering  effort  should  reach  a  minimum  from  this  point  until  the  end  of  the 
contract.  An  additional  milestone  of  note  is  the  beginning  of  a  follow-on  contract.  For  D&V 
this  would  be  the  beginning  of  the  EMD  contract;  for  EMD  it  would  be  the  production 
contract.  It  was  initially  postulated  that  the  beginning  of  a  follow-on  would  coincide  with  a 
reduction  in  staff.  While  this  may  be  true  at  an  aggregate  level,  this  was  not  observed  for 
Development  Engineering. 

The  spending  rate  per  period  of  time  was  computed  for  each  system  and  the  ratio  of 
each  period's  rate  to  the  PDR-to-CDR  period  was  computed  to  obtain  the  hypothesis 
coefficients,  a/  The  median  values  by  program  phase  were  chosen  as  the  preferred 
coefficients.  Tne  resuiting  values  are  shown  graphically  in  Figure  3-4,  In  both  of  these 
figures,  two  sequences  of  milestones  are  shown.  The  first  set  is  for  those  situations  where 
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Figure  3-4 
Values  for  a; 


CDR  preceded  the  first  flight.  The  second  set  is  for  those  situations  where  first  flight 
preceded  CDR. 

3.6  TECHNICAL  COMPONENT  DEVELOPMENT 

For  use  in  this  portion  of  the  hypothesized  methodology,  estimating  relationships  were 
explored  that  would  compute  the  design  basis  cost  or  the  peak  design  effort.  Note  that  the 
peak  design  effort  according  to  our  working  definition  is  the  "PDR-toCDR"  cost-per-month. 
The  database  from  was  put  into  a  statistics  package,  CO$TAT,  and  regressions  were  run.  The 
dependent  variables  examined  were  Development  Engineering  Peak,  Design  Basis,  and 
Development  Engineering.  The  better  predictive  relationships  were  indeed  those  for 
Development  Engineering  Peak,  Rmax,  which  led  to  increased  confidence  in  our  hypothesis. 

Table  3-1  presents  the  equations  evaluated  which  estimate  the  peak  expenditure  rate 
of  Development  Engineering  cost  in  millions  of  FY88  dollars.  Parameters  evaluated  include 
the  cost  of  prototype  manufacturing  (PM),  duration  in  months  (TD),  missile  weight  in 
pounds  (MW),  prototype  quantity  (PQ),  program  phase  dummy  (D1  and  D2),  cost  of  first 
prototype  (PCFU),  mission  type  dummy  (MI,  M2,  and  M3),  and  an  air  vehicle  completeness 
dummy  (C2  and  C3).  The  program  phase  stratification  was  by  D&V  (0,0),  EMD  (1,0),  and 
PIP  (0,1).  The  mission  type  was  by  surface-to-air  (1,0,0),  air-to-air  (0,1,0),  other  tactical 
(0,0,1),  or  strategic/cruise  (0,0,0).  The  Air  Vehicle  completeness  stratification  was  by  entire 
air  vehicle  (0,0),  guidance  section  (1,0),  and  other  section  (0,1). 

Table  3-1 

Peak  Equations  Evaluated 


« 

EQUATION 

r2 

SE 

■V 

F 

COMMENTS 

1 

0.1656  (PM  °’5375) 

42 

2,25 

15 

11 

All  diU  points. 

2 

-0.3038+(0.0386  PM) 

93 

0,73 

15 

186 

All  diu  points. 

II 

IRIIliitBHHHHHHHi 

47 

2.69 

15 

13 

All  diU  points 

4 

1.1129  (PCFU  a5133)  (MW  *0  0212) 

43 

2.79 

15 

6 

All  d*u  points. 

5 

0.6125  (PCFU  °'5433)  (3.2529  D1)  (0.9194  D2) 

72 

2.29 

15 

13 

All  data  points. 

6 

0.6286(PCFU°'5171)  (3,1495D1) 

64 

3 

12 

11 

Psmafinrft  ^  Imc  PIP  t*n«rinvr 

7 

0  7935  (PCFU  °'5185)  (1.9831  D1)  (0.690602) 

91 

03 

13 

41 

Equations  1  through  6  are  against  the  entire  data  set.  Equation  2  has  a  good  fit  but 
the  negative  intercept  may  make  it  not  useful  for  lower  cost  systems.  Also,  the  use  of  total 
prototype  manufacturing  is  not  quite  as  satisfying  as  using  the  cost  of  one  prototype  in 
predicting  the  peak.  Introducing  missile  weight  as  a  further  explainer  of  missile  complexity 
(Equation  4)  did  not  help  nor  did  stratifying  by  mission  type  (Equation  5).  However  since 
there  were  only  three  points  in  the  PIP  category  and  these  exhibited  scatter,  we  removed 
these  in  Equation  6.  Removing  two  outliers  in  Equation  7  provided  for  a  good  fit.  We  feel 
justified  removing  these  two  particular  points  as  one  had  a  well-publicized  overrun  which 
was  not  reflected  on  the  cost  reports  and  the  second  program  has  been  redirected  often. 
Equation  7  is  the  recommended  equation.  This  is  shown  graphically  in  Figure  3-5. 

3.7  COMBINED  EQUATIONS 

Combining  the  peak  equation  from  Section  3.6  (Rmax)  with  the  schedule  coefficients 
of  Section  3.5  (a,)  lead  to  the  recommended  method  shown  in  Figure  3-6.  Six  equations  are 
presented  which  represent  three  program  phases  and  two  sequences  of  milestones. 


DEDemVal  =  (0.67Di  +1D2  +  0.72D3  +  0.36D4  +  0.02D5)  (0.7935  PCFU0-5185) 
DeEMD  =  (0.54DJ  +1D2  +  0.67D3  +  0.50D4  +  0.1 1D5)  (1 .574PCFU0-5185) 
DEPIP  =  (0.67D!  +1D2  +  0.72D3  +  0.36D4  +  0.02D5)  (0.5480PCFU°-5185) 


where: 

DEj^mVal =  D&V  Development  Engineering  in  Millions  of  FY88  Dollars 

DEemd  =  EMD  Development  Engineering  in  Millions  of  FY88  Dollars 

DEpjp  =  PIP  Development  Engineering  in  Millions  of  FY88  Dollars 

Dj  =  Months  from  Contract  Award  to  PDR 

D2  =  Months  from  PDR  to  CDR 

D3  =  Months  from  CDR  to  First  Flight 

D4  =  Months  from  First  Flight  to  Last  Flight 

D5  =  Months  from  Last  Flight  to  Contract  End 

PCFU  =  Prototype  Cost  of  First  Unit  in  Millions  of  FY88  Dollars 

Alternatives  where  First  Flight  precedes  CDR 

DEnemVal  =  (0.67Dj  +1D2  +  0.47D3  +  0.02D4)  (0.7935  PCFU0-5185) 
DEemd  =  (0.54Dj  +1D2  +  0.54D3  +  0.11D4)(1.574PCFU°-5185) 
DEp,p  =  (0.67D,  +1D2  +  0.47D3  +  0.02D4)  (0.5480  PCFU0  5185) 

where: 

D3  =  Months  from  CDR  to  Last  Flight 

D4  =  Months  from  Last  Flight  to  Contract  End 


Figure  3-6 

Combined  Equations 
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4 

EXTENSIONS  AND  ALTERNATIVES 


Many  estimating  situations  require  that  the  estimator  make  further  adjustments  to  a 
CER  due  to  cost  influencing  issues  that  are  not  addressed  by  the  CER’s  independent  variables 
These  issues  are  usually  unique  and  require  subjective  adjustments  be  made  to  a  CER.  The 
Development  Engineering  CER  presented  in  Section  3  is  particularly  versatile  and  may  be 
readily  adapted  to  the  use  of  such  adjustments.  This  section  discusses  some  variations  and 
alternative  uses  of  this  paper's  equations  and  data. 

4.1  ADJUSTMENTS  TO  TERMS 

The  recommended  equations  from  Section  3  are  comprised  of  two  components  which 
may  be  separately  tweaked  or  utilized  independently.  Consider  some  alternative  ways  to  use 
the  schedule  component.  First,  the  coefficients  on  the  months  for  the  duration  terms  may  be 
adjusted.  One  example  of  doing  this  would  be  to  examine  the  values  in  the  database  and 
select  an  analogous  system's  coefficients.  Another  example  may  be  to  subjectively  adjust  any 
single  period  as  deemed  appropriate.  For  example,  if  a  program  is  expected  to  compete  two 
contractors  early  in  a  development  program  and  down  select  at  PDR,  then  multiplying  the 
"award  to  PDR"  term  by  a  value  of  two  would  be  an  appropriate  adjustment.  Another 
adjustment  to  the  schedule  component  is  if  a  system's  design  will  be  relatively  stable  at  some 
point  in  time  other  than  CDR,  then  that  date  may  be  used  in  lieu  of  the  actual  CDR  date. 

Now  consider  some  alternative  ways  to  use  the  peak  component  of  Section  3,7's 
equations.  In  lieu  of  computing  the  peak  expenditure  rate  from  the  CER,  the  peak  term  may 
be  obtained  by  selecting  analogous  system  values.  The  user  may  wish  to  obtain  the  peak  value 
by  entirely  different  means.  This  may  be  via  a  build-up  of  personnel  required  to  design  the 
system  or  by  scaling  from  one  of  the  values  from  the  database.  Also,  for  an  on-going 
contract,  an  estimate  at  completion  may  be  obtained  from  this  method  by  extrapolating  from 
an  early  month’s  actual  expenditure  rate. 

4.2  ADJUSTMENTS  PARAMETER 

Another  subjective  issue  that  relates  to  estimating  Development  Engineering  is  design 
complexity,  design  heritage  and  programmatics.  In  the  recommended  methodology  of  Section 
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3-7,  these  issues  are  indirectly  addressed  as  they  already  influence  the  value  of  the  independent 
parameter,  the  prototype  cost.  Even  so,  the  final  effort  in  this  study  was  to  determine  if  our 
predictive  ability  could  be  improved  using  subjective  adjustments  to  account  for  these  issues. 
The  adjustment  parameter  was  hypothesized  to  be  a  measure  of  design  challenge  and 
programmatic  complexity.  The  adjustment  variable  will  be  the  product  of  two  variables.  One 
for  complexity  and  one  for  programmatics.  The  value  of  each  would  vary  about  unity  such 
that  the  nominal  case  would  be  equal  to  one  and,  therefore,  have  no  effect  on  the  total  cost. 
These  are  computed  based  upon  analyst  judgment  on  several  distinct  issues. 

Several  design  complexity  issues  were  identified  that  were  believed  would  further 
explain  variation  in  the  data.  The  first  issue,  design  challenge  is  the  analyst's  opinion  of  the 
complexities  involved  in  the  design  of  an  item  based  upon  technology,  tolerances,  parts  count, 
etc.  The  packaging  issue  is  to  account  for  systems  where  space  is  a  premium  and  that 
extensive  design  effort  is  required  to  meet  volume  constraints.  Countermeasures  is  an  issue 
for  systems  w’ich  require  innovative  design  in  order  to  be  responsive  to  severe 
countermeasures,  discrimination,  or  evolving  countermeasure  threat.  The  heritage  issue  is  a 
measure  cf  the  existence  of  similar  existing  designs  or  technology  that  may  be  leveraged  upon 
to  simplify  the  design  tasks  of  tills  system.  These  design  complexity  issues  are  quantified  by 
use  of  a  subjective  scheme. 

Several  programmatic  complexity  issues  were  identified  that  were  believed  would 
further  explain  variation  in  the  data.  The  number  of  platforms  for  which  the  system  is  being 
designed  was  considered  a  programmatic  issue.  It  was  believed  to  be  more  programmatic 
than  technical  because  additional  design  costs  are  realized  through  further  coordination 
imposed  upon  the  designers,  such  as  technical  interchange  meetings,  etc.  The  number  of 
government  agencies  is  a  measure  of  the  number  of  customers  to  which  the  design  contractor 
must  report,  or  the  number  of  services  for  which  design  issues  must  be  addressed.  The 
number  of  subcontractors  is  an  issue  because  an  increased  number  of  subcontractors  increases 
the  design  effort  and  interfaces  may  become  more  challenging  and,  once  again,  increase 
coordination  activities.  These  programmatic  complexity  issues  are  quantified  by  use  of 
counting  the  occurrences  of  these  characteristics. 


This  hypothesis  was  applied  to  our  database  of  monthly  systems.  The  Issues  were 
quantified  as  depicted  in  Figure  4-1.  For  the  sake  of  simplicity,  the  technical  issue  values  were 
limited  to  +/-  0.10.  It  is  believed  that  since  the  nominal  value  for  these  are  1.0,  future 
researchers  or  users  may  wish  to  add  further  adjustments.  For  that  reason,  a  category  of 
adjustment  called  "additional  adjustments"  is  shown  in  Figure  4-1.  For  this  example,  the 
technical  issues  were  applied  at  the  subsystem  level  and  were  weighted  by  their  respective 
subsystem  hardware  costs. 

Table  4-1  presents  the  peak  equation  that  was  computed  for  this  hypothesis.  Equation 
1  utilized  all  the  data  points.  It  represents  a  dramatic  improvement  over  Equation  5  in  Table 
3-1  which  was  also  for  all  15  data  points.  Removing  the  one  outlier  in  Equation  2  gave  the 
best  fit  statistics  of  any  peak  equation,  Removing  an  additional  outlier  (Equation  3)  gave  a 
little  less  fit  than  Equation  2  but  a  lower  standard  error.  Equation  3  is  directly  comparable  to 
Equation  7  from  Table  3-1  as  shown  in  Figure  4-2.  Note  that  the  F  value  has  increased, 
indicating  that  introduction  of  the  Adjustment  variable  increases  the  regression's  significance. 
The  r^  is  improved  but  the  standard  error  has  not  improved.  The  residuals  were  somewhat 
better,  exhibiting  a  lower  average  enor.  The  Table  3-1  equation  predicted  eight  points  within 
20%  while  this  new  equation  predicted  ten  points  within  20%.  Note  the  exponent  on  the 
Adjustment's  variable  in  Equation  3  is  lower  than  that  of  Equations  1  and  2.  This  difference  is 
due  to  the  influence  of  one  particular  data  point  that  was  discarded  as  an  outlier  which  had  the 
highest  adjustment  value  in  the  data  set.  The  large  exponent  makes  all  of  these  equations  very 
sensitive  to  the  value  of  Adjustments  and  since  this  is  mostly  a  subjective  value,  the  user  is 
cautioned  when  using  any  of  these  three  equations.  For  this  reason.  Equation  3  is  suggested 
as  the  most  conservative  with  respect  to  the  Adjustments  variable.  Note  also  that  the  values 
used  in  this  regression  ranged  from  0.92  to  1.13  and  that  the  application  of  values 
considerably  beyond  this  range  may  provided  extreme  results.  If  the  reader  chooses  to  apply 
this  equation  to  an  estimating  problem,  the  subjective  assessments  must  be  sensibly  and 
defensibly  applied. 
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Rmax  =  /(PCFU,  Dl,  D2,  Adjustment) 


Adjustments  =  subjective  complexity  variable  where  1.0  is  the  typical  norm 

Adjustments  =  technical  complexity  *  program  complexity 
(weighted  by  subsystem  if  desired) 

Technical  complexity  =  1  + 


+  0.10 

Challenging  design 

-  0.10 

Extremely  simple  design 

+  0.10 

Packaging  considerations  (i.e.,  small  area) 

+  0.10 

High  countermeasure  threat 

-  0.10 

Good  heritage 

±n 

Additional  complexity  adjustments  as  user  deems 
appropriate 

Program  complexity  =  1  + 

+  (.02  *  Xj)  where  Xj  is  the  number  of  platforms  beyond  1 
on  which  weapon  will  operate. 

+  (.02  *  X2)  where  X2  is  the  number  of  government  agencies 
running  the  program  beyond  1 . 

+  (.01  *  X3)  where  X3  is  the  number  of  major  subcontractors. 
±  n  additional  programmatic  adjustments  as  user 

_ deems  appropriate. 

Figure  4-1 

Adjustment  Issue  Quantification 
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Table  4-1 

Equations  with  Adjustments  Evaluated 


• 

EQUATION 

r2 

ST. 

H 

H 

COMMENTS 

1 

0.6030PCFU°-341JADJ5-26932.2844D10.5902D2 

85 

0.65 

15 

21 

All  dau  points. 

2 

0.7341PCFUO'3852AD15o4751.9222D10.5141D2 

97 

0.48 

14 

91 

Equation  #1  less  ERINT. 

3 

0.7458PCPU°-403C)ADJ4-23071.8971D!0.5353D2 

95 

0.41 

13 

Without  Adjustments:  Peak  =  0.7935  (Proto_CFU)0-5185  1.983lD1  0.6906D2 
With  Adjustments:  Peak  =  0.7458  (Proto„CFU)04030  (ADJ*-2307  i.8971D1  0.5353d2 


Where:  D1,D2  = 

0,0  Dem  Val 

1,0  EMD 

0,1  PIP 

WilhQul 

mb 

r2 

90.89 

95.00 

F 

40.92 

58.03 

SE 

0.4012 

0.2272 

n 

13 

13 

df 

9 

9 

Figure  4-2 

Peak  Equation  Comparison 
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5 

CONCLUSION 


5.1  RESULTS 

A  primary  methodology  for  estimating  Air  Vehicle  Development  Engineering  was 
developed.  This  method  utilizes  schedule  milestones  and  the  cost  of  the  first  prototype.  It  is 
applicable  to  D&V,  EMD,  or  PIP  programs.  This  method  is  very  versatile  as  discussed  in 
Section  4.  The  schedule  terms  and  the  peak  terms  are  independent  so  that  user-supplied 
alternatives  may  be  substituted  for  either  if  desired.  Also,  parameters  to  address  subjective 
issues  such  as  design  complexity,  heritage,  and  programmatics  were  analyzed.  It  is 
recommended  that  Development  Engineering  cost  of  other  types  of  systems  and  equipment  be 
addressed  with  this  type  of  method.  Also  the  use  of  milestones  could  be  investigated  for 
applicability  to  CERs  for  all  other  R&D  cost  accounts. 
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